Image processing device, method of processing image, image processing program, and imaging device

ABSTRACT

An information processing system configured to perform predetermined processing based on out-of-focus information corresponding to a first area of an image, the out-of-focus information corresponding to a difference in focus between the first area of the image and a second area of the image that is different from the first area of the image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation Application of application Ser. No.16/557,240, filed Aug. 30, 2019 which is a Continuation Application ofapplication Ser. No. 14/894,435, filed Nov. 27, 2015, which issued asU.S. Pat. No. 10,440,251 on Oct. 8, 2019 which is a 371 Nationalizationof PCT/JP2014/070143, filed Jul. 22, 2014 and claims the benefit ofJapanese Priority Patent Application JP 2013-152937 filed on Jul. 23,2013, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to an image processing device, a methodof processing an image, an image processing program, and an imagingdevice.

BACKGROUND ART

In recent years, imaging devices, such as digital cameras, have beenwidely spread, and there are some imaging devices provided with afunction to automatically and optimally adjust focus and exposureaccording to an image capturing scene. However, special stage effects,such as adjustment of a composition and shading off of a background atthe time of capturing an image, are still largely depending on skills ofusers. Especially, for beginners who do not have knowledge of cameras,it is difficult to capture an image with the special staging effects.Therefore, in recent years, many technologies to shade off thebackground, and the like have been proposed (PTL 1).

CITATION LIST Patent Literature

[PTL 1]

JP 2008-233470A

SUMMARY

Many of related technologies detect an object, and set a diaphragmaccording to information of the object. To be specific, the technologiesdetect a face of a person, and set a diaphragm according to informationof the size, position, direction or the like of the face. Such adiaphragm determination technique is automatically controlled, and thusthere is a possibility that an image is captured against intension ofthe user.

The present technology has been made in view of such a problem, andthere is a need for providing an image processing device, a method ofprocessing an image, an image processing program, and an imaging devicethat enable easy acquisition of a blurred image by performing processingbased on information related to blur in the image.

Solution to Problem

To solve the above problem, a first technology is an informationprocessing system configured to perform predetermined processing basedon out-of-focus information corresponding to a first area of an image,the out-of-focus information corresponding to a difference in focusbetween the first area of the image and a second area of the image thatis different from the first area of the image.

Advantageous Effects of Invention

According to an embodiment of the present technology, processing isperformed based on information related to blur in an image, whereby theuser can easily acquire a blurred image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imageprocessing device according to an embodiment of a present technology.

FIG. 2 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a first embodiment.

FIG. 3 is a diagram for describing a method of calculating a blurprediction amount.

FIGS. 4A and 4B are diagrams illustrating display forms in the firstembodiment.

FIG. 5 is a diagram illustrating a display form in the first embodiment.

FIG. 6 is a diagram illustrating a display form in the first embodiment.

FIG. 7 is a flowchart illustrating a flow of processing in the firstembodiment.

FIG. 8 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a second embodiment.

FIGS. 9A to 9C are diagrams illustrating display forms in the secondembodiment.

FIG. 10 is a flowchart illustrating a flow of processing in the secondembodiment.

FIG. 11 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a third embodiment.

FIGS. 12A and 12B are diagrams illustrating display forms in the thirdembodiment.

FIGS. 13A and 13B are diagrams illustrating display forms in the thirdembodiment.

FIG. 14 is a flowchart illustrating a flow of processing in the thirdembodiment.

FIG. 15 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a fourth embodiment.

FIGS. 16A to 16C are diagrams illustrating display forms in the fourthembodiment.

FIG. 17 is a flowchart illustrating a flow of processing in the fourthembodiment.

FIG. 18 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a fifth embodiment.

FIGS. 19A and 19B are diagrams illustrating display forms in the fifthembodiment.

FIG. 20 is a flowchart illustrating a flow of processing in the fifthembodiment.

FIG. 21 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a sixth embodiment.

FIGS. 22A and 22B are diagrams illustrating display forms in the sixthembodiment.

FIG. 23 is a flowchart illustrating a flow of processing in the sixthembodiment.

FIG. 24 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a seventh embodiment.

FIGS. 25A and 25B are diagrams illustrating display forms in the seventhembodiment.

FIG. 26 is a flowchart illustrating a flow of processing in the seventhembodiment.

FIG. 27 a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto an eighth embodiment.

FIGS. 28A and 28B are diagrams describing segmentation of an image inthe eighth embodiment, and FIG. 28C is a diagram illustrating ahistogram used for composition determination.

FIGS. 29A to 29C are diagrams describing a second example of imagesegmentation in the eighth embodiment.

FIG. 30 is a flowchart illustrating a flow of processing in the eighthembodiment.

FIG. 31 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a ninth embodiment.

FIGS. 32A to 32D are diagrams illustrating images acquired by diaphragmbracket image capturing in the ninth embodiment.

FIGS. 33A to 33D are diagrams illustrating a second example of an imageacquired by the diaphragm bracket image capturing in the ninthembodiment.

FIG. 34 is a flowchart illustrating a flow of processing in the ninthembodiment.

FIG. 35 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a tenth embodiment.

FIGS. 36A and 36B are diagrams illustrating an example an image to whichimage processing has been applied in the tenth embodiment.

FIG. 37 is a flowchart illustrating a flow of processing in the tenthembodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present technology will be describedwith reference to drawings. Note that description will be given in thefollowing order.

<1. First Embodiment>

[1-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[1-2. Processing in an image processing device and an imaging device]

<2. Second Embodiment>

[2-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[2-2. Processing in an image processing device and an imaging device]

<3. Third Embodiment>

[3-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[3-2. Processing in an image processing device and an imaging device]

<4. Fourth Embodiment>

[4-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[4-2. Processing in an image processing device and an imaging device]

<5. Fifth Embodiment>

[5-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[5-2. Processing in an image processing device and an imaging device]

<6. Sixth Embodiment>

[6-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[6-2. Processing in an image processing device and an imaging device]

<7. Seventh Embodiment>

[7-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[7-2. Processing in an image processing device and an imaging device]

<8. Eighth Embodiment>

[8-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[8-2. Processing in an image processing device and an imaging device]

<9. Ninth Embodiment>

[9-1. Configurations of an image processing device and an imaging deviceincluding the image processing device]

[9-2. Processing in an image processing device and an imaging device]

<10. Tenth Embodiment>

[10-1. Configurations of an image processing device and an imagingdevice including the image processing device]

[10-2. Processing in an image processing device and an imaging device]

<11. Modifications>

1. First Embodiment

[1-1. Configurations of an Image Processing Device and an Imaging DeviceProvided with the Image Processing Device]

First, configurations of an image processing device 1 and an imagingdevice 1100 provided with a function of the image processing device 1according to a first embodiment will be described. FIG. 1 is a blockdiagram illustrating the image processing device 1.

The image processing device 1 is configured from an attention areadetermination unit 11, a blur information acquisition unit 12, and ablur related processing unit 13. The image processing device 1 isoperated in an imaging device, for example.

FIG. 2 is a block diagram illustrating a configuration of the imagingdevice 1100 provided with a function of the image processing device 1.The imaging device 1100 is configured from an optical imaging system1001, a lens control unit 1002, an imaging element 1003, a preprocessingcircuit 1004, a camera processing circuit 1005, an image memory 1006, adisplay unit 1007, an input unit 1008, a storage unit 1009, and acontrol unit 1010. Further, the control unit 1010 functions as an objectdetection unit 1011, a focus determination unit 1012, and an attentionarea determination unit 11, a blur information acquisition unit 12, anda blur related processing unit 13, which configure the image processingdevice 1. In the first embodiment, the blur related processing unit 13functions as a graph creation unit 14 and a display control unit 15.

The optical imaging system 1001 is configured from an image capturinglens for collecting light from an object to the imaging element 1003, adrive mechanism for moving the image capturing lens and performingfocusing and zooming, a shutter mechanism, an iris mechanism, and thelike. These lens and mechanisms are driven by control of the lenscontrol unit 1002. An optical image of the object obtained through theoptical imaging system 1001 is formed on the imaging element 1003.

The lens control unit 1002 is a microcomputer in a lens, for example,and controls operations of the drive mechanism, the shutter mechanism,and the iris mechanism of the optical imaging system 1001, and the like,according to control from the control unit 1010 and the image adjustmentunit 12. This enables adjustment of an exposure time (shutter speed), adiaphragm value (F-number), and the like.

The imaging element 1003 photoelectrically converts incident light froman object to convert it into a charge amount, and outputs the chargeamount as an analog imaging signal. The analog imaging signal outputfrom the imaging element 1003 is output to the preprocessing circuit1004. As the imaging element 1003, a charge coupled device (CCD), acomplementary metal oxide semiconductor, or the like is used.

The preprocessing circuit 1004 samples and holds the imaging signaloutput from the imaging element 1003 by correlated double sampling (CDS)processing to favorably maintain a signal/noise (S/N) ratio. Further,the preprocessing circuit 1004 controls a gain by auto gain control(AGC) processing, performs analog/digital (A/D) conversion, and outputsa digital image signal. These pieces of processing are performedaccording to control from the control unit 1010. The camera processingcircuit 1005 applies, to an image signal from the preprocessing circuit1004, signal processing, such as white balance adjustment processing,color correction processing, gamma correction processing, Y/C conversionprocessing, and auto exposure (AE) processing.

The image memory 1006 is a volatile memory, for example, a buffer memoryconfigured from a dynamic random access memory (DRAM). The image memory1006 temporarily stores image data to which predetermined processing hasbeen applied by the preprocessing circuit 1004 and the camera processingcircuit 1005.

The display unit 1007 is a display device configured from a liquidcrystal display (LCD), a plasma display panel (PDP), an organic electroluminescence (EL) panel, or the like, for example. In the display unit1007, a through image during imaging, a captured image recorded in thestorage unit 1009, and the like are displayed.

The input unit 1008 is made of a power button for switching powerON/OFF, a release button for instructing start of recording of animaging image, an operator for zoom adjustment, a touch screenintegrally configured with the display unit 1007, and the like. When aninput is given to the input unit 1008, a control signal according to theinput is generated and output to the control unit 1010. The control unit1010 then performs arithmetic processing and control corresponding tothe control signal.

The storage unit 1009 is a mass storage medium, such as a hard disk, amemory stick (a registered trademark of Sony Corporation), or an SDmemory card. An image is stored in a compressed state based on astandard, such as Joint Photographic Experts Group (JPEG), for example.Further, exchangeable image file format (EXIF) data includinginformation related to the stored image and additional information suchas a date and time of imaging is also stored in association with theimage.

The control unit 1010 is configured from a central processing unit(CPU), a random access memory (RAM), and a read only memory (ROM), andthe like. In the ROM, programs read and operated by the CPU are stored.The RAM is used as a work memory of the CPU. The CPU executes varioustypes of processing according to the programs stored in the ROM, andcontrols the entire imaging device 1100 by issuing commands.

Further, the control unit 1010 functions as the object detection unit1011, the focus determination unit 1012, and the attention areadetermination unit 11, the blur information acquisition unit 12, and theblur related processing unit 13, which configure the image processingdevice 1, by executing predetermined programs. However, theseconfigurations are not only realized by the programs, but may also berealized by a combination of a dedicated device by hardware that has theabove functions, a circuit, and the like.

Note that an object to be processed of the image processing device 1 inthe first embodiment is a frame image that configures a through image.

The object detection unit 1011 detects an object from an image to beprocessed. An example of the object includes a face of a person, forexample. Note that a thing detected by the object detection unit 1011does not necessarily need to be a face of a person, and may be ananimal, a building, or the like, as long as it can be detected. As adetection method, template matching based on a shape of an object,template matching based on luminance distribution of a face, a methodbased on a characteristic amount of a flesh-colored portion or a face ofa human included in an image, or the like, can be used. Further,detection accuracy may be enhanced by a combination of these techniques.

The focus determination unit 1012 determines whether an object detectedin an image is focused. Examples of a factor to focus an object includehalf press of a shutter by a user, full press of the shutter, autofocus, continuous auto focus (AF), manual focus, and the like. Adetermination result by the focus determination unit 1012 is supplied tothe attention area determination unit 11.

The attention area determination unit 11 determines an area to be paidattention (hereinafter, called attention area) in an image to beprocessed. The attention area is an area in which a thing that isconsidered as a principal object in the image by a user, who capturesthe image, such as a person, an animal, a plant, and a building, exists.

The attention area determination unit 11 determines the attention areabased on object information acquired by the object detection unit 1011,for example. Further, the attention area determination unit 11 maydetermine an object focused by the imaging device 1100 as the attentionarea. Further, the attention area determination unit may determine theattention area based on an input that specifies an area on a throughimage from the user (an input to a touch panel, various buttons, or thelike included in the imaging device 1100). Further, the attention areadetermination unit may determine an object closer to a center in animage as the attention area. Further, the attention area determinationunit may determine an object having a predetermined size or more in animage as the attention area.

Information that indicates the attention area in an image determined bythe attention area determination unit 11 is supplied to the blurinformation acquisition unit 12. The blur information acquisition unit12 acquires blur information in an area outside the attention area(hereinafter, called non-attention area) in the image. The blurinformation includes a blur amount and a blur prediction amount. Theblur amount can be acquired based on a depth of field or phasedifference range-finding information in a state where the attention areais focused in the imaging device 1100. The degree of blur is larger asthe value of the blur amount is larger. The blur prediction amountindicates to what extent the non-attention area is blurred on theassumption that the attention area is focused. Here, the blur predictionamount as the blur information will be described with reference to FIG.3. The blur prediction amount can be expressed by the followingexpression (1):

Blur prediction amount=(A−B)/d . . . (1) where an image forming positionof the attention area is A, an image forming position of thenon-attention area is B, and a focal depth is d.

In this expression (1), only the defocus amount (A−B) of the attentionarea and the background area may just be obtained. Therefore, it is notnecessary to actually form an image on the attention area, and thedefocus amount can be obtained from various AF systems, such as acontrast detection system and a phase difference system, and an AFsensor, and the like. The degree of blur in an image may become largeras the value of the blur prediction amount is larger.

Note that the attention area and the non-attention area are notnecessarily areas having a uniform distance, and for example, theseareas may be obtained and treated such that the areas are classified andtreated for each given defocus amount, the areas may be averaged andtreated for each vicinity area on an XY plane, or the widest and thesame distance areas may be treated as representative values.

Further, the focal depth is determined based on a permissible circle ofconfusion δ, and δ differs according to an appreciation size. Therefore,a reference may be determined by a method of selecting the reference bythe user in advance (according to A3/A4/L size printing, what inchequivalent of the screen size, or the like), or a method ofautomatically selecting the reference according to the number ofrecording pixels of an image. Further, when trimming is performed, thefocal depth can be treated in an appropriate unit by multiplying acoefficient according to a trimming amount. The blur informationacquired by the blur information acquisition unit 12 is supplied to theblur related processing unit 13. The blur related processing unit 13performs various types of processing according to the blur information.In the first embodiment, the blur related processing unit 13 functionsas the graph creation unit 14 and the display control unit 15. The blurrelated processing unit 13 presents the blur information to the user bydisplaying a graph that indicates the blur information in the displayunit 1007 included in the imaging device 1100.

The graph creation unit 14 creates a graph that indicates the blurinformation based on the blur information supplied from the blurinformation acquisition unit 12. The display control unit 15 performsdisplay control of superimposing and displaying the graph generated bythe graph creation unit 14 on the image in the display unit 1007 of theimaging device 1100.

FIGS. 4A and 4B are first examples of forms of the graph generated bythe graph creation unit 14 and displayed in the display unit 1007. Inthese examples, the blur amount as the blur information is presentedwith a bar graph superimposed and displayed on the through imagedisplayed in the display unit 1007.

The blur amount and the height of the bar graph correspond to eachother, and the bar graph becomes higher when the blur amount is large,and the bar graph becomes shorter when the blur amount is small.Further, a color of the bar graph may be changed according to the heightof the bar graph so that the graph can be easily recognized by the user.Note that the presentation of the blur amount is not limited to the bargraph, and may be a pie graph.

In the example of FIG. 4A, the person is the attention area, and thebackground is the non-attention area. The non-attention area is not muchblurred. Therefore, the blur amount is small, and the bar graphdisplayed on the left on the screen is short.

Meanwhile, in the example of FIG. 4B, the person is the attention area,and the background other than the person is the non-attention area. Thenon-attention area is blurred. Therefore, the blur amount is large, andthe bar graph is high.

The blur information is visually presented in this way, whereby the usercan accurately recognize to what extent the non-attention area isblurred at present. Note that it is favorable that the blur informationacquisition unit 12 continuously acquire the blur amount during imagecapturing, and the blur related processing unit changes the graphdisplay of the blur amount in real time based on the blur mount acquiredby the blur information acquisition unit 12 on a steady basis.

FIG. 5 is a diagram illustrating a second example of graph display ofthe blur amount. In FIG. 5, a graph is displayed based on the blurprediction amount instead of the blur amount. In the bar graph of FIG.5, the height of the bar graph indicates a range of the blur amount thatcan be realized by a present attention area, a focal distance, and amaximum aperture of a lens. Further, with the mark superimposed anddisplayed in a bar graph manner, a blur prediction amount in the presentF-number is indicated.

FIG. 6 is a third embodiment of a graph generated by the graph creationunit 14, and displayed in the display unit 1007. In the thirdembodiment, the graph creation unit 14 creates a histogram of the blurprediction amount supplied form the blur information acquisition unit12.

In the histogram, the horizontal axis represents the magnitude of theblur amount or the blur prediction amount. The blur amount or the blurprediction amount becomes larger from the left to the right. Further,the vertical axis of the histogram represents an area ratio of a blurredarea and a non-blurred area in an image. By presenting the blurinformation with a histogram, more detailed information can be presentedto the user. The image processing device 1 and the imaging device 1100having a function of the image processing device 1 according to thefirst embodiment are configured as described above.

[1-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 1and the imaging device 1100 will be described with reference to FIG. 7.FIG. 7 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied to the control unit 1010and the image processing device 1, and the focal distance and theF-number have been acquired. First in step S101, AF start processing isperformed in the imaging device 1100. The AF start processing isexecuted by various methods, such as a half press operation of ashutter, which is one of the input unit 1008, a touch operation to thetouch panel, and automatic start by detection of object change. Further,regarding a phase difference sensor, and the like, which can acquiredistance information on a steady basis, an AF function may be operatedon a steady basis without a specific input operation from the user. Whenthe AF start processing is performed, a predetermined control signal issupplied to the image processing device 1, and processing by the controlunit 1010 and the image processing device 1 is started.

Next in step S102, an object in an image is detected by the objectdetection unit 1011. Next, in step S103, an attention area and anon-attention area in the image are determined by the attention areadetermination unit 11. As described above, the attention area is aperson, for example, and the non-attention area is an area other thanthe person as the attention area, such as a background. Next, in stepS104, whether the attention area is focused is determined by the focusdetermination unit 1012. A determination result by the focusdetermination unit 1012 is supplied to the blur information acquisitionunit 12. Next, in step S105, a blur amount or a blur prediction amountin the image is acquired by the blur information acquisition unit 12.Next, in step S106, a graph indicating the blur information is generatedby the graph creation unit 14. Then, in step S106, the graph issuperimposed and displayed on a through image in the display unit 1007by the display control unit 15. The first embodiment of the presenttechnology is configured as described above. According to the firstembodiment, the degree of blur of an image to be acquired by imagecapturing can be presented to the user in various forms in aneasy-to-understand manner. Accordingly, even in a case where the degreeof blur of an image in a through image is hard to understand because thesize of the display unit 1007 of the imaging device 1100 is small, theuser can easily recognize the degree of blur.

2. Second Embodiment

[2-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a second embodiment of the present technology will be described.In the second embodiment, an image processing device 2 is also operatedin an imaging device 1200. FIG. 8 is a block diagram illustrating aconfiguration of the imaging device 1200 provided with the imageprocessing device 2 according to the second embodiment.

The image processing device 2 according to the second embodiment isdifferent from the first embodiment in that a blur related processingunit 23 functions as an enlargement processing unit 24 and a displaycontrol unit 25. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted.

An object to be processed of the image processing device 2 in the secondembodiment is a frame image that configures a through image. The secondembodiment presents the degree of blur of an image to the user in aneasy-to-understand manner by enlarging a part of the image.

The enlargement processing unit 24 generates an image in which a rangeincluding both of an attention area and a non-attention area in theimage is enlarged. For example, when a through image is in a stateillustrated in FIG. 9A, a range in which both of the attention area andthe non-attention area are included is enlarged as illustrated in FIG.9B. An enlargement range is favorably to set such that ratios of theattention area and the non-attention area become nearly equal in theenlargement range so that a difference of the degree of blur between theattention area and the non-attention area. However, a method of settingthe enlargement range is not limited to the above example.

The display control unit 25 superimposes and displays the image enlargedby an enlargement processing unit on the through image, as illustratedin FIG. 9B. Further, a segmented enlarged image may be fully displayedin a display unit 1007, as illustrated in FIG. 9C. Further, asillustrated in FIG. 9B, the enlarged image is superimposed and displayedon the through image, and when there is an input to the enlarged image(a contact of a finger to the touch panel), the enlarged image may befully displayed in the display unit 1007, as illustrated in FIG. 9C.

Note that this enlargement display is favorably displayed for a giventime after completion of an AF operation of the imaging device 1200.Note that the blur related processing unit 23 determines whether a blurprediction amount obtained by a blur information acquisition unit 12 isa predetermined amount or more. When the blur prediction amount is thepredetermined amount or more, the blur related processing unit 23 causesthe enlargement processing unit 24 and the display control unit 25 toexecute the above enlargement display processing.

[2-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 2and the imaging device 1200 will be described with reference to FIG. 10.FIG. 10 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S201, imageenlargement processing is performed by the enlargement processing unit24. Then, in step S202, an enlarged image is superimposed and displayedon a through image by the display control unit 25.

The second embodiment of the present technology is configured asdescribed above. The second embodiment enlarges and displays a part ofan image such that both of the attention area and the non-attention areain the image are included, whereby the user can more accuratelyrecognize the degree of blur.

3. Third Embodiment

[3-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a third embodiment of the present technology will be described. Inthe third embodiment, an image processing device 3 is also operated inan imaging device 1300. FIG. 11 is a block diagram illustrating aconfiguration of the imaging device 1300 provided with a function of theimage processing device 3 according to the third embodiment.

The image processing device 3 according to the third embodiment isdifferent from the first embodiment in that a blur related processingunit 33 functions as a blur amount adjustment unit 34 and a displaycontrol unit 35.

Configurations other than the above are similar to those in the firstembodiment, and thus description is omitted. An object to be processedof the image processing device 3 in the third embodiment is a frameimage that configures a through image. In the third embodiment, a bluramount adjustment function by the blur amount adjustment unit 34 isautomatically started, and a blur adjustment user interface is furtherautomatically displayed.

The blur amount adjustment unit 34 adjusts a diaphragm of a lens andchanges a blur amount of a non-attention area by transmitting apredetermined control signal to a lens control unit 1002 of the imagingdevice 1300 according to an input to an input unit 1008 from the user.

When the blur amount of an image is adjusted by the blur amountadjustment unit 34 according to an input from the user, a bluradjustment user interface is displayed in a display unit 1007 by thedisplay control unit 35 so that the user can easily perform an input.

When a focal distance and an F-number are obtained, and an adjustablerange of a blur prediction amount is a predetermined amount or more bychanging the F-number, the blur related processing unit automaticallystarts a blur amount adjustment function by the blur amount adjustmentunit. The display control unit 35 then displays the blur amountadjustment user interface in the display unit 1007.

FIGS. 12A and 12B are diagrams illustrating states in which the userinterface of the blur amount adjustment function is displayed in thedisplay unit 1007. In the examples of FIGS. 12A and 12B, a pointer 200movable in a vertical direction by an operation of the user isdisplayed. A position of the pointer 200 is moved in conjunction withthe blur amount of an image.

The blur amount of an image is changed according to the position of thepointer 200. At an upper end of a bar 210 that indicates a moving pathof the pointer 200, an icon is displayed together with characters of“clear” to capture the non-attention area clearly without shading offthe area. Further, at a lower end of the bar 210 that is the moving pathof the pointer 200, an icon is displayed together with characters of“shade off” to blur and capture the non-attention area. When the usermoves the pointer 200 upward, the diaphragm becomes smaller, and thenon-attention area becomes clear with a less blur amount. Meanwhile,when the user moves the pointer 200 downward, the diaphragm becomeslarger, and the non-attention area is blurred with a large blur amount.In FIG. 12A, the pointer 200 is positioned at the upward clear side, andthus the image is not much blurred. Meanwhile, in FIG. 12B, the pointer200 is positioned at the downward shading-off side, and thus the bluramount of the image is large.

Note that, as illustrated in FIG. 13A, a position that indicates arecommended blur amount is displayed, and display of an arrow, blinkingof the position, display of animation, and the like may be performed inorder to induce the user to move the pointer 200 to the position. Thisenables the user who does not understand a blur control function toeasily blur an image.

Further, as illustrated in FIG. 13B, an area other than the adjustablerange by the blur control is filled, and when the present degree ofadjustment by the blur control has reached a limit, the pointer 200 maybe blinked for a given time, and tips may be displayed. In the exampleof FIG. 13B, a tip to emphasize the degree of blur of an image isdisplayed.

Note that the form of the user interface of the blur control function isnot limited to the one illustrated in FIGS. 12A and 12B, and 13A and13B. Any interface may be employed as long as the interface can adjust ablur amount of an image.

[3-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 3and the imaging device 1300 will be described with reference to FIG. 14.FIG. 14 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S301, the blurrelated processing unit 33 causes the blur amount adjustment function bythe blur amount adjustment unit 34 to automatically start. Note that, asdescribed above, the blur amount adjustment function is favorablyautomatically started when the adjustable range of the blur predictionamount is a predetermined amount or more by changing the F-number. Next,in step S302, the display control unit 35 then displays the blur amountadjustment user interface in the display unit 1007. The third embodimentof the present technology is configured as described above. According tothe third embodiment, the blur amount adjustment function isautomatically started, whereby the user can easily perform adjustment ofthe degree of blur. Further, a user who does not know the existence ofthe blur amount adjustment function or a user who knows the blur amountadjustment function but has a low frequency of use can be urged to usethe blur amount adjustment function.

4. Fourth Embodiment

[4-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a fourth embodiment of the present technology will be described.In the fourth embodiment, an image processing device 4 is also operatedin an imaging device 1400. FIG. 15 is a block diagram illustrating aconfiguration of the imaging device 1400 provided with a function of theimage processing device 4 according to the fourth embodiment.

The fourth embodiment is different from the first embodiment in that ablur related processing unit 43 functions as an image processing unit44, a display control unit 45, and a blur amount adjustment unit 46.

Configurations other than the above are similar to those in the firstembodiment, and thus description is omitted. An object to be processedof the image processing device 4 in the fourth embodiment is a frameimage that configures a through image. The fourth embodiment presentswhat a blurred image is like to the user by displaying the image inwhich a blur amount has been adjusted in a display unit 1007.

A blur information acquisition unit 12 acquires a blur prediction amountby present setting of the imaging device 1400, and a blur predictionamount in a maximum aperture.

The image processing unit 44 generates an image in which the degree ofblur in a state where the F-number is opened is reproduced by applyingblur processing to an image. When the blur prediction amount in thepresent setting is a predetermined amount or less, and the blurprediction amount in the maximum aperture is a predetermined amount ormore, the image processing unit 44 generates an image in which thedegree of blur in a state where the F-number is opened is reproduced.The blur amount adjustment unit 46 is similar to the blur amountadjustment unit in the third embodiment.

The display control unit 45 makes transition of display of a throughimage in the display unit 1007 from the state of FIG. 16A to the stateof FIG. 16B. In the state of FIG. 16B, an image acquired in the presentsetting of the imaging device 1400 and an image to which imageprocessing has been applied into a state in which the F-number is openedby the image processing unit 44 are displayed side by side. In FIG. 16B,The left-side image is the present image and the right-side image is theimage to which image processing has been applied into a state in whichthe F-umber is opened. Accordingly, the user can easily compare thepresent image (through image) and the image in which the degree of bluris emphasized. Note that the display form of the images is not limitedto the one illustrated in FIG. 16B, and the images may be arranged inthe vertical direction. Further, the present image and a plurality ofimages to which processing has been applied by the image processing unit44 may be displayed so that a difference occurs in the degree of blur.

Then, when any of the images is selected by the user, the displaycontrol unit 45 fully displays the selected image in the display unit1007, as illustrated in FIG. 16C. As a method of selecting an image,there are a method by selection with an operation key and a method inwhich a finger is brought into contact with any of the images in a casewhere the input unit 1008 is a touch panel. When the selected image isfully displayed in the display unit 1007, the display control unit 45may display the blur amount adjustment function user interface by theblur amount adjustment unit 46 in the display unit 1007. This enablesthe user to acquire an easily selected image by image capturing.Further, the blur amount adjustment function is automatically started,whereby the user can adjust the blur amount such that the image furthersuits to own taste from the state of the selected image. Note that, inthe state where a plurality of images is displayed, as illustrate inFIG. 16B, when any of the images is selected by the user, the image maybe acquired by automatic image capturing. This enables even a user whodoes not know the operation of the blur control to capture a blurredimage.

[4-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 4and the imaging device 1400 will be described with reference to FIG. 17.FIG. 17 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S401, an image towhich image processing has been applied into a state in which theF-number is opened is generated by the image processing unit 44.

Then, in step S402, an image acquired in the present setting of theimaging device 1400, and an image to which processing has been appliedinto a state in which the F-number is opened by the image processingunit 44 are displayed in the display unit 1007 by the display controlunit 45.

The fourth embodiment of the present technology is configured asdescribed above. According to the fourth embodiment, the user can easilycompare the present image and the image in which the degree of blur isemphasized.

Accordingly, whether it is better to acquire a blurred image by imagecapturing, and the like, can be easily determined.

5. Fifth Embodiment

[5-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a fifth embodiment of the present technology will be described. Inthe fifth embodiment, an image processing device 5 is also operated inan imaging device 1500. FIG. 18 is a block diagram illustrating aconfiguration of the imaging device 1500 provided with a function of theimage processing device 5 according to the fifth embodiment.

The image processing device 5 according to the fifth embodiment isdifferent from the first embodiment in that a blur related processingunit 53 functions as a composition determination unit 54, a recommendedblur amount determination unit 55, and a display control unit 56.Configurations other than the above are similar to those in the firstembodiment, and thus description is omitted.

An object to be processed of the image processing device 5 in the fifthembodiment is a frame image that configures a through image. The fifthembodiment determines a composition in an image, and applies mask(zebra) display processing to the image according to a recommended bluramount in the composition. The composition determination unit 54determines whether the image matches a predetermined composition. Thecomposition determination unit 54 includes a plurality of algorithms forcomposition determination in advance, and executes the compositiondetermination processing based on the plurality of compositiondetermination algorithms. The composition determination unit 54determines, for example, whether an image includes a composition, suchas a whole body shot, a bust shot, close-up of a face, or the like,based on the position of an object such as a person, the size of theperson, the size of the face, or the like. Further, the compositiondetermination unit 54 determines whether the image corresponds to athirds composition, a fourths composition, a sun flag composition, adiagonal composition, or the like, based on a predetermined compositiontemplate. A determination result by the composition determination unit54 to the recommended blur amount determination unit 55. The recommendedblur amount determination unit 55 determines a recommended blur amountcorresponding to the composition of the image determined by thecomposition determination unit 54. This recommended blur amount is arecommended blur amount in the composition corresponding to eachcomposition. The recommended blur amount is obtained by referring to atable in which a composition and a recommended blur amount areassociated, for example. For example, as illustrated in FIG. 19A, whenit has been determined by the composition determination unit that animage is a thirds composition, the recommended blur amount determinationunit 55 determines the recommended blur amount corresponding to thecomposition. When a difference between the recommended blur amount andthe blur prediction amount is within a predetermined range, the displaycontrol unit 56 superimposes and displays a mask on the image such thatthe image becomes to have the recommended blur amount, as illustrated inFIG. 19B. This enables the user to easily recognize the state in whichan image has a recommended blur amount.

[5-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 5and the imaging device 1500 will be described with reference to FIG. 20.FIG. 20 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S501, a compositionof a present image is determined by the composition determination unit54. Next, in step S502, a recommended blur amount of the determinedcomposition is determined by the recommended blur amount determinationunit 55. Then, in step S503, when a difference between the recommendedblur amount and the blur prediction amount is within a predeterminedrange, a mask is superimposed and displayed on the image such that theimage becomes to have the recommended blur amount by the display controlunit 56.

The fifth embodiment of the present technology is configured asdescribed above. According to the fifth embodiment, a blur amount of animage can be emphasized by mask display, whereby the user can easilyrecognize what an image having a large blur amount is like.

6. Sixth Embodiment

[6-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a sixth embodiment of the present technology will be described. Inthe sixth embodiment, an image processing device 6 is also operated inan imaging device 1600. FIG. 21 is a block diagram illustrating aconfiguration of the imaging device 1600 provided with a function of theimage processing device 6 according to the sixth embodiment.

The image processing device 6 according to the sixth embodiment isdifferent from the first embodiment in that a blur related processingunit 63 functions as a recommended composition determination unit 64, agraph creation unit 65, a display control unit 66, and an imagesegmentation unit 67. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted.

An object to be processed of the image processing device 6 in the sixthembodiment is a frame image that configures a through image. The sixthembodiment presents, to the user, blur information in a recommendedcomposition in the image, with a histogram by the image processingdevice 6.

The recommended composition determination unit 64 includes a pluralityof algorithms for detecting a composition from an image, and executescomposition detection processing based on the plurality of compositiondetection algorithms.

The recommended composition determination unit 64 determines, forexample, whether an image includes a composition, such as a whole bodyshot, a bust shot, close-up of a face, or the like, based on theposition of an object such as a person, the size of the person, the sizeof the face, or the like. Further, the recommended compositiondetermination unit 64 determines whether the image includes a thirdscomposition, a fourths composition, a sun flag composition, a diagonalcomposition, or the like, based on a predetermined composition template.The recommended composition determination unit 64 then determines arecommended composition (hereinafter, referred to as recommendedcomposition) based on the position and the size of an object in eachcomposition, and the like.

The blur information acquisition unit 12 acquires a blur predictionamount of the entire image, and calculates a blur prediction amount in anon-attention area in the recommended composition determined by therecommended composition determination unit 64. The calculated blurprediction amount is supplied to the graph creation unit 65.

The graph creation unit 65 is similar to the graph creation unit of thefirst embodiment, and creates a histogram indicating the supplied blurinformation. The display control unit 66 displays the histogram createdby the graph creation unit in the display unit. The display control unit66 superimposes and displays the blur prediction amount on the image asa histogram in a display unit 1007, as illustrated in FIG. 22A, forexample. Note that, when the blur prediction amount is displayed by thehistogram, a frame indicating the recommended composition is favorablydisplayed together with the histogram, as illustrated in FIG. 22A. Thisenables the user to easily recognize the position and the size of therecommended composition.

Further, when the histogram is displayed, a histogram indicating theblur prediction amount in the recommended composition and a histogram inthe entire image are favorably displayed together, as illustrated inFIG. 22B. In FIG. 22B, as an example of the display, the solid lineindicates the blur prediction amount in the entire image, and the dashedline indicates the blur prediction amount in the recommendedcomposition. The histograms are displayed in this way, the user caneasily recognize the blur prediction amount in the entire image and theblur prediction amount in the recommended composition. The imagesegmentation unit 67 performs processing (trimming) of segmenting arange of the recommended composition determined by the recommendedcomposition determination unit 64 from the image. With the segmentationby the image segmentation unit, the image can have a more favorablecomposition.

[6-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 6and the imaging device 1600 will be described with reference to FIG. 23.FIG. 23 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S104, object detection, determination of anattention area, and focus determination are performed after AF startprocessing. Then, in step S105, a blur prediction amount of anon-attention area in the entire image is acquired by a blur informationacquisition unit 12.

Next, in step S601, a recommended composition is determined by therecommended composition determination unit 64. Next, in step S602, ablur prediction amount of the non-attention area in the recommendedcomposition is acquired by the blur information acquisition unit 12.Then, in step 603, a histogram indicating blur information is created bythe graph creation unit. Then, in step S604, the histogram is displayedin the display unit 1007 by the display control unit 66, as illustratedin FIGS. 22A and 22B.

Next, in step S605, a range of the recommended composition is segmentedfrom the image and a trimming image is generated by the imagesegmentation unit 67. Note that the segmentation of the image by theimage segmentation unit 67 may be automatically performed when therecommended composition is determined by the recommended compositiondetermination unit 64, or may be performed only when an instructioninput of the image segmentation is given from the user. The sixthembodiment of the present technology is configured as described above.The sixth embodiment presents the blur prediction amount in therecommended composition with a histogram while presenting therecommended composition, whereby the user can easily recognize what thedegree of blur of the image in the recommended composition is like.

7. Seventh Embodiment

[7-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a seventh embodiment of the present technology will be described.In the seventh embodiment, an image processing device 7 is also operatedin an imaging device 1700. FIG. 24 is a block diagram illustrating aconfiguration of the imaging device 1700 provided with a function of theimage processing device 7 according to the seventh embodiment.

In the image processing device 7 according to the seventh embodiment, ablur related processing unit 73 functions as a recommended compositiondetermination unit 74, a recommended blur amount determination unit 75,a graph creation unit 76, a display control unit 77, and an imagesegmentation unit 78.

An object to be processed of the image processing device 7 in theseventh embodiment is a frame image that configures a through image. Theseventh embodiment presents a recommended blur amount in a recommendedcomposition and a blur prediction amount in the recommended compositionto the user with graph display. The recommended compositiondetermination unit 74 is similar to that in the sixth embodiment, anddetermines a recommended composition that is a composition ofrecommendation from an image. A determination result of the recommendedcomposition determination unit 74 is supplied to the recommended bluramount determination unit 75 and is also supplied to a blur informationacquisition unit 12. The blur information acquisition unit 12 acquires ablur prediction amount in the entire image, and also acquires a blurprediction amount in the recommended composition.

The recommended blur amount determination unit 75 determines arecommended blur amount corresponding to the recommended compositiondetermined by the recommended composition determination unit 74. Therecommended blur amount is a recommended blur amount in the composition,corresponding to each composition. The recommended blur amount is, forexample, obtained by referring to a table in which a composition and arecommended blur amount are associated.

In the seventh embodiment, first, a recommended composition in an imageis determined by the recommended composition determination unit 74.Next, the recommended blur amount determination unit 75 determines arecommended blur amount in the recommended composition. For example,when the determined recommended composition is so-called a whole bodyshot, in which the whole body of a person that is an attention areaappears, the recommended blur amount determination unit 75 determines arecommended blur amount that a blur prediction amount of a non-attentionarea is favorably small (blur prediction amount<first blur threshold).

Further, when the determined recommended composition is close-up of aface of the person that is the attention area, the recommended bluramount determination unit 75 determines a recommended blur amount thatthe blur amount of the non-attention area is favorably large (thirdthreshold<blur amount).

Further, when the determined recommended composition is so-called a bustshot of the person that is the attention area, the recommended bluramount determination unit determines a recommended blur amount that theblur amount of the non-attention area is favorably about between thewhole body shot and the close up of the face (second blur threshold<bluramount<third blur threshold). Note that a relationship among the firstto third thresholds is as described in the following expression 1:First blur threshold<Second blur threshold<Third blur threshold  (1)

The determined recommended composition and the recommended blur amountare supplied to the graph creation unit 76. The graph creation unit 76creates a graph like the one illustrated in FIGS. 25A and 25B. In theexample of the graphs of FIGS. 25A and 25B, the height of a bar graphindicates a range of the blur amount that can be realized by a presentattention area, a focal distance, and a maximum aperture of a lens.Further, a mark superimposed and displayed in a bar graph mannerindicates a blur prediction amount in the present F-number.

Further, a frame that indicates a range is displayed on the bar graph,whereby a recommended blur amount in the composition is indicated. Therecommended blur amount can present a range of the recommended bluramount by being displayed with the frame. Further, an icon presents ablur prediction amount of when an image is trimmed by the recommendedcomposition. Accordingly, the user can easily recognize what kind ofphotograph having what blur amount can be acquired by capturing thephotograph by what kind of composition. Further, the blur predictionamount in the entire image may be presented with an icon.

Further, as illustrated in FIG. 25B, when the recommended blur amountfalls outside the range of the blur amount that can be realized, tipsindicating how the blur amount is increased may be displayed by acharacter string. The image segmentation unit 78 performs processing ofsegmenting the range of the recommended composition determined by therecommended composition determination unit 74, from the image. With thesegmentation by the image segmentation unit 78, the image can have amore favorable composition.

[7-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 7and the imaging device 1700 will be described with reference to FIG. 26.FIG. 26 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S701, a recommendedcomposition is determined by the recommended composition determinationunit 74. Next, in step S702, a blur prediction amount in the recommendedcomposition is acquired by the blur information acquisition unit 12.Next, in step S703, a recommended blur amount in the recommendedcomposition is determined by the recommended blur amount determinationunit 75. Next, in step S704, a graph for presenting the blur predictionamount, the recommended blur amount, and the blur prediction amount inthe recommended composition is created by the graph creation unit 76.Next, in step S705, the graph is displayed in a display unit 1007 by thedisplay control unit 77.

Then, in step S706, the range of the recommended composition issegmented from the image by the image segmentation unit 78, and atrimming image is generated. Note that the segmentation of the image bythe image segmentation unit 78 may be automatically performed when therecommended composition is determined by the recommended compositiondetermination unit 74, or may be performed only when an instructioninput of the image segmentation is given from the user.

The seventh embodiment of the present technology is configured asdescribed above. According to the seventh embodiment, a recommended bluramount in a recommended composition and a blur prediction amount in therecommended composition are presented with graph display, whereby theuser can capture an image using these pieces of information as areference.

8. Eighth Embodiment

[8-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, an eighth embodiment of the present technology will be described.In the eighth embodiment, an image processing device 8 is also operatedin an imaging device 1800. FIG. 27 is a block diagram illustrating aconfiguration of the imaging device 1800 provided with a function of theimage processing device 8 according to the eighth embodiment.

The image processing device 8 according to the eighth embodiment isdifferent from the first embodiment in that a blur related processingunit 83 functions as a segmentation range determination unit 84, arecommended blur amount determination unit 85, an image segmentationunit 86, a display control unit 87, and an enlargement processing unit88. Configurations other than the above are similar to those in thefirst embodiment, and thus description is omitted.

An object to be processed of the image processing device 8 in the eighthembodiment is a captured image acquired by an imaging device. The eighthembodiment performs segmentation processing (trimming) of the imageaccording to blur information in the image.

By trimming the captured image, the image can have a more favorablecomposition. However, a segmentation range has various shapes andpositions. Therefore, the eighth embodiment determines a segmentationrange such that the degree of blur in the image stands out according toblur information.

Segmentation of an image will be described with reference to FIGS. 28Aand 28B using a thirds composition as an example. First, thesegmentation range determination unit 84 sets a plurality of candidatesof a segmentation range by a thirds composition. For example, thecandidates of the segmentation range can be determined such that a spacehaving a predetermined size is formed in a direction where anon-attention area exists in the image.

As illustrated in FIG. 28A, a plurality of candidates exists in size andposition in a thirds composition, for example. In FIG. 28A, threesegmentation ranges are illustrated as examples of the candidates. Notethat the composition is not limited to a thirds composition. A pluralityof other composition data, such as a golden ratio composition, is storedin advance as templates, and the composition is favorably selected basedon selection of the user or other conditions. Alternatively, forexample, when a segmentation range extends outside the image in a thirdscomposition, the composition may be changed to a golden ratiocomposition.

When having determined the segmentation range candidates, thesegmentation range determination unit 84 supplies composition candidateinformation to the blur information acquisition unit 12 and therecommended blur amount determination unit 85. The blur informationacquisition unit 12 acquires a blur prediction amount of eachsegmentation range candidate, and supplies the blur prediction amount tothe segmentation range determination unit 84. Further, the recommendedblur amount determination unit 85 determines a recommended blur amountin each segmentation range candidate. The recommended blur amountdetermination unit 85 is similar to that of the seventh embodiment. Therecommended blur amount of each segmentation range candidate is suppliedto the segmentation range determination unit 84.

The segmentation range determination unit 84 determines a segmentationrange from the plurality of segmentation range candidates based on threecriteria: a composition where the blur prediction amount is closest tothe recommended blur amount; a position where the blur prediction amountin a segmentation range is closest to the recommended blur amount; and aposition where the degree of separation of the blur prediction amountsof the attention area and the non-attention area is highest. Note that aweight of each criterion may be changed for each composition.

Whether the degree of separation of the blur amounts is high can bedetermined by showing the blur prediction amounts of the attention areaand the non-attention area in each composition as histograms, asillustrated in FIG. 28C. In the example of FIG. 28C, among compositionsa, b, and c, the attention area and the non-attention area are mostseparated in the composition a. The composition a has a small trimmingarea, and thus the blur amount becomes largest when the image isenlarged. Further, an area where the blur amount is small (the portionof the face) and an area where the blur amount is large (the background)are clearly separated in the composition a, and thus the composition ahas a steep mountain like area, as illustrate d by the histogram of FIG.28C. The size of the trimming area of the composition b is almost thesame as that of the composition a. However, the larger blur area sideincludes middle to large wide ranges of blur amounts. Therefore, thecomposition b has a gently sloping mountain like area, as illustrated bythe histogram of FIG. 28C. The composition c has a large trimming area,and an emphasized blur amount is smaller than those of the compositionsa and b. A larger blur amount maintain exists in the middle of thehorizontal axis. Note that the composition a, the composition b, and thecomposition c are compositions as specific examples set for convenienceof description.

Therefore, when the composition a is selected as the segmentation range,and the segmentation of the image is executed, the image becomes the oneillustrated in FIG. 28B. By performing the segmentation processing of acaptured image in this way, the image can have a more favorablecomposition, and the degree of blur of the non-attention area can befurther emphasized.

Note that, when a diaphragm bracket function is turned ON, a diaphragmis changed suitable for each composition candidate and bracket imagecapturing may be performed. Here, a case in which one segmentation rangeis to be determined from among a plurality of composition candidatesbased on the three criteria, and the recommended blur amounts of theplurality of compositions are almost the same will be described. Imagesegmentation is performed with respect to all of the segmentation rangesof the plurality of compositions having almost the same recommended bluramounts, and trimming images are favorably presented to the user. Forexample, as illustrated in FIG. 29A, when there are two compositioncandidates of a composition d and a composition e, and the recommendedblur amounts of these two compositions are almost the same, trimming isperformed for each of the compositions d and e. FIG. 29B illustrates atrimming image based on the composition d, and FIG. 29C is a trimmingimage based on the composition e.

Then, the plurality of types of trimming images is presented to theuser, and the user may be prompted to select any of the trimming images.This enables the user to acquire an image having the degree of blur thatsuits to own taste.

Note that the segmentation range of the image may be variable withrespect to an aspect of an original image. For example, when the aspectof the original image is “4:3”, the segmentation range may be changeableto “3:2” or “16:9” according to characteristics of the object. Theenlargement processing unit 88 performs processing of enlarging thetrimming image to an original image size using a super-resolutiontechnique when storing the trimming image in the storage unit 1009.Accordingly, the image has a more emphasized degree of blur than beforethe trimming.

[8-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 8and the imaging device 1800 will be described with reference to FIG. 30.FIG. 30 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S104, object detection, determination of anattention area, and focus determination are performed after AF startprocessing. Then, in step S801, a segmentation range candidate isdetermined by the segmentation range determination unit 84. Next, instep S802, a blur prediction amount of the segmentation range candidateis acquired by the blur information acquisition unit 12. Next, in stepS803, a recommended blur amount of the segmentation range candidate isdetermined by the recommended blur amount determination unit 85.

Then, in step S804, the segmentation range of the image is determined bythe segmentation range determination unit 84. Segmentation rangeinformation is supplied to the image segmentation unit 86. Next, in stepS805, the segmentation range is segmented from the image by the imagesegmentation unit 86.

Next, in step S806, the trimming image is enlarged to an original imagesize by the enlargement processing unit 88 using a super-resolutiontechnique, and is stored in the storage unit 1009. Then, a segmentedimage is displayed in a display unit 1007 by the display control unit87, as illustrated in FIG. 28B, or FIG. 29B or 29C.

Accordingly, the user can confirm the segmented image. Note that, whenthe segmented image is displayed in the display unit 1007, the image towhich the enlargement processing has been applied by the enlargementprocessing unit 88 is favorably displayed.

The eighth embodiment of the present technology is configured asdescribed above. According to the eighth embodiment, segmentation of animage is performed based on blur information in the image, whereby animage in which the degree of blur is emphasized can be created. Further,an image can be segmented in a range considering blurs in a principalobject (attention area) and a background (non-attention area), and animpressive segmented image, which is different from a composition thatincludes the entire object area, can be provided.

9. Ninth Embodiment

[9-1. Configurations of an Image Processing Device and an Imaging DeviceProvided with a Function of the Image Processing Device]

Next, a ninth embodiment of the present technology will be described. Inthe ninth embodiment, an image processing device 9 is also operated inan imaging device 1900. FIG. 31 is a block diagram illustrating aconfiguration of the image processing device 9 according to the ninthembodiment.

The image processing device 9 according to the ninth embodiment isdifferent from the first embodiment in that a blur related processingunit 93 functions as an imaging operation control unit 94 and a displaycontrol unit 95. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted. Anobject to be processed of the image processing device 9 in the ninthembodiment is a frame image that configures a through image. The imagingoperation control unit 94 controls execution of a diaphragm bracketimage capturing mode by transmitting a predetermined control signal tothe lens control unit 1002. The diaphragm bracket image capturing modeis an image capturing mode to acquire a plurality of images havingdifferent diaphragm aperture diameters by a single image capturingoperation. A plurality of images is acquired by changing the aperturediameters of a diaphragm, and thus a plurality of images havingdifferent blur amounts can be acquired at single image capturing.

When a diaphragm bracket is executed in a direction of making theF-number larger than the present setting, a plurality of images havingsmaller blur amounts in a non-attention area than a present image can beacquired, as illustrated in FIGS. 32B to 32D. In FIGS. 32A to 32D, FIG.32A illustrates an image in present setting, and the F-number becomeslarger and the images have smaller blur amounts in the non-attentionarea, in order of FIGS. 32B, 32C, and 32D.

Further, when the diaphragm bracket is executed in a direction of makingthe F-number smaller than the present setting, a plurality of imageshaving larger blur amounts in the non-attention area than the presentimage can be acquired, as illustrated in FIGS. 33B to 33D. In FIGS. 33Ato 33D, FIG. 33A illustrates an image in the present setting, and theF-number becomes smaller and the images have larger blur amounts in thenon-attention area, in order of FIGS. 33B, 33C, and 33D.

Further, the F-number is changed before and after the present setting ofthe F-number, and the diaphragm bracket can be executed. Accordingly, animage having a larger blur amount and an image having a smaller bluramount than the present image can be acquired at single image capturing.

The display control unit 95 displays the image acquired by the diaphragmbracket image capturing in a display unit 1007.

[9-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 9and the imaging device 1900 will be described with reference to FIG. 34.FIG. 34 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S901, the imagingoperation control unit 94 activates the diaphragm bracket imagecapturing mode according to the blur amount. The imaging operationcontrol unit 94 executes the diaphragm bracket in a direction of makingthe F-number larger than the present setting when the blur amount is apredetermined amount or more. Accordingly, a plurality of images havingsmaller blur amounts in the non-attention area than the present imagecan be acquired, as illustrated in FIGS. 32B to 32D. Further, theimaging operation control unit 94 executes the diaphragm bracket in adirection of making the F-number smaller than the present setting whenthe blur amount is a predetermined amount or less. Accordingly, asillustrated in FIGS. 33B to 33D, a plurality of images having largerblur amounts in the non-attention area than the present image can beacquired.

Next, in step S902, recording processing of the plurality of imagesacquired by the diaphragm bracket image capturing is performed by thecontrol unit 1010. The captured image is, for example, stored in astorage unit 1009 in a compressed state based on a standard such asJPEG. Note that the plurality of images captured by the diaphragmbracket image capturing may be associated and stored by tagging or thelike as mutually associated images. Further, exchangeable image fileformat (EXIF) data including information related to the stored image andadditional information such as a date and time of imaging may also bestored in association with the image. Then, in step S903, the capturedimage captured by the diaphragm bracket is displayed in the display unit1007 as a preview image by the display control unit 95. Accordingly, theuser can instantly confirm the image captured by the diaphragm bracket.

The ninth embodiment of the present technology is configured asdescribed above. According to the ninth embodiment, the diaphragmbracket image capturing is automatically executed according to the blurinformation. Therefore, for a user who does not know the existence ofthe diaphragm bracket function or a user who knows the diaphragm bracketfunction but has a low frequency of use, an image acquired by thediaphragm bracket image capturing can be provided.

10. Tenth Embodiment

[10-1. Configurations of an Image Processing Device and an ImagingDevice Provided with a Function of the Image Processing Device]

Next, a tenth embodiment of the present technology will be described. Inthe tenth embodiment, an image processing device 10 is also operated inan imaging device 2000. FIG. 35 is a block diagram illustrating aconfiguration of the imaging device 2000 provided with a function of theimage processing device 10 according to the tenth embodiment.

The image processing device 10 according to the tenth embodiment isdifferent from the first embodiment in that a blur related processingunit 103 functions as an image processing unit 104 and a display controlunit 105. Configurations other than the above are similar to those inthe first embodiment, and thus description is omitted. An object to beprocessed of the image processing device 1 in the tenth embodiment is aframe image that configures a through image. The tenth embodimentapplies shading off processing to a captured image captured by theimaging device 2000. This enables an image having a less blur amount atthe time of capturing to be an image having a large blur amount.

The image processing unit 104 applies the shading off processing to anon-attention area of the captured image based on known image shadingoff algorithm. By applying the shading off processing to thenon-attention area of the image, the image is changed from a stateillustrated in FIG. 36A to a state illustrated in FIG. 36B. In FIG. 36B,the background other than a person as an attention area is blurred,compared with FIG. 36A. When a blur amount of the non-attention area isa predetermined amount or more, the blur related processing unit 103causes the image processing unit 104 to execute background shading offprocessing. The display control unit 105 displays an image to which theshading off processing has been applied by the image processing unit 104in a display unit 1007.

[10-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 10and the imaging device 2000 will be described with reference to FIG. 37.FIG. 37 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S1001, whether a fullpress operation of a shutter has been performed by the user isdetermined. The acquisition of blur information in step S105 and thedetermination of step S1001 are repeated until the full press operationof a shutter is performed (No in step S1001).

When the full press operation of a shutter has been performed by theuser, the processing proceeds to step S1002 (Yes in step S1001). Next,in step S1002, image capturing processing is performed in the imagingdevice 2000, and an image is acquired. Next, in step S1003, whether ablur amount of when the image is captured is a predetermined value orless is determined.

When the blur amount is the predetermined value or less, the processingproceeds to step S1004 (Yes in step S1003).

Then, in step S1004, shading off processing is applied to anon-attention area of the captured image by the image processing unit104. By applying the shading off processing to the image when the bluramount is the predetermined amount or less, the image processing unit104 can make the degree of blur of the non-attention area of the imagehaving a less degree of blur large. Accordingly, the user can easilyobtain an image having a blur background.

Next, in step S1005, recording processing of the image to which theshading off processing has been applied is performed. The captured imageis, for example, stored in a storage unit 1009 in a compressed statebased on a standard such as JPEG under control of a control unit 1010.Further, EXIF data including additional information, such as informationrelated to the stored image, and a date and time of imaging may also bestored in association with the image.

Then, in step S1006, the captured image to which the shading offprocessing has been applied is displayed in the display unit 1007 as apreview image by the display control unit 105. Accordingly, the user caninstantly confirm the image to which the shading off processing has beenapplied.

Description is returned to step S1003. In step S1003, when it has beendetermined that the blur amount is the predetermined about or less, theprocessing proceeds to step S1006 (No in step S1003). Then, in stepS1006, the captured image is displayed in the display unit 1007 as apreview image by the display control unit 105. Note that, in this case,steps S1004 and S1005 have not been performed, and thus the imagedisplayed in the display unit 1007 is an image to which the shading offprocessing has not been applied.

The tenth embodiment is configured as described above. According to thetenth embodiment, even the user who is not familiar with an operation ofa camera and has difficulty in capturing a blur image can easily acquirea blur image. Further, even in a case where it is difficult to acquire ablur image by diaphragm control of the imaging device, a blur image canbe acquired. According to an embodiment of the present technology,instinctive and easy-to-understand display can be provided even in anenvironment where visual observation is difficult with a liquid crystalpanel, and the like. The above-described first to tenth embodiments, theprocessing performed by the image processing devices can be executed byhardware or software. When processing by software is executed, a programin which a processing sequence is recorded is installed into a memory ofa control unit of an imaging device and executed.

For example, the program can be recorded in a recording medium, such asa hard disk or a ROM, in advance. Alternatively, the program can berecorded in a recording medium, such as a compact disc read only memory(CD-ROM), a digital versatile disc (DVD), or a semiconductor memory.Such a recording medium can be provided as packaged software. The userinstalls the packages software in the imaging device.

Note that the program that is provided in the Internet as an applicationcan be transferred and installed to the imaging device, in addition toinstallation from the above-described recording medium to the imagingdevice.

11. Modification

As described above, embodiments of the present technology have beenspecifically described. However, the present technology is not limitedto these embodiments, and various modifications based on the technicalidea of the present technology is possible.

In the above-described embodiments, description has been given taking acase of applying the present technology to an image acquired in animaging device as an example. However, the present technology is alsoapplicable to a moving image.

Further, the above first to tenth embodiments may be combined andimplemented. For example, the graph display of blur information in thefirst embodiment and the enlargement display in the second embodimentare combined, and a graph is displayed while a part of an image isenlarged and displayed.

Further, the object is not limited to a person or a human of a person,and the above-described processing may be performed using a dog, a cat,a flower, a dish, or the like, as the object, using a dog/cat facedetection function, a flower detection function, a dish detectionfunction, or the like.

Further, the present technology may employ the following configurations.

(1) An information processing system including: circuitry configured toperform predetermined processing based on out-of-focus informationcorresponding to a first area of an image, the out-of-focus informationcorresponding to a difference in focus between the first area of theimage and a second area of the image that is different from the firstarea of the image.

(2) The system of (1), wherein the circuitry is configured to: identifythe first area of the image; and acquire the out-of-focus informationcorresponding to the first area of the image.

(3) The system of any of (1) to (2), wherein the second area of theimage is an in-focus area of the image.

(4) The system of (1) to (3), wherein the system includes: a firstinformation processing apparatus; and a second information processingapparatus configured to be connected to the first information processingapparatus.

(5) The system of any of (1) to (4), wherein the predeterminedprocessing includes outputting a notification corresponding to theout-of focus information.

(6) The system of any of (1) to (5), wherein the predeterminedprocessing includes controlling a display to display a notificationindicating an out-of-focus amount based on the out-of-focus information.

(7) The system of (6), wherein the circuitry is configured to controlthe display to display the image and the notification simultaneously.

(8) The system of any of (6) to (7), wherein the circuitry is configuredto control the display to display the image with the notificationsuperimposed on the displayed image.

(9) The system of any of (1) to (8), wherein the predeterminedprocessing includes controlling a display to display a bar graphindicating an out-of-focus amount based on the out-of-focus information.

(10) The system of (9), wherein a height of the bar graph corresponds toa range of an out-of-focus amount that may be realized based on at leastone of the second area, a focal distance and a maximum aperture of alens capturing the image.

(11) The system of any of (1) to (10), wherein the predeterminedprocessing includes controlling a display to display an enlarged versionof the image including the first area and the second area.

(12) The system of (11), wherein an enlargement range of the enlargedversion of the image is set so that a display area corresponding to thefirst area and a display area corresponding to the second area aresubstantially similar.

(13) The system of any of (11) to (12), wherein the circuitry isconfigured to control the display to display the image and the enlargedversion of the image simultaneously.

(14) The system of any of (11) to (13), wherein the circuitry isconfigured to control the display to display the enlarged version of theimage superimposed on the image.

(15) The system of any of (1) to (14), wherein the predeterminedprocessing includes controlling a display to display the image whileapplying a graphical effect to the first area of the image.

(16) The system of any of (1) to (15), wherein the predeterminedprocessing includes trimming the image based on the out-of-focusinformation.

(17) The system of (16), wherein the circuitry is configured to controla display to display the trimmed image.

(18) The system of any of (1) to (17), wherein the predeterminedprocessing includes generating a first trimmed version of the image anda second trimmed version of the image based on the out-of-focusinformation.

(19) The system of any of (1) to (18), wherein the predeterminedprocessing includes generating a first trimmed version of the image anda second trimmed version of the image based on the out-of-focusinformation.

(20) The system of (19), wherein the circuitry is configured to controla display to display the first trimmed version of the image and thesecond trimmed version of the image.

(21) The system of (4), wherein the first information processingapparatus is configured to: identify the second area of the image;acquire the out-of-focus information; and transmit the out-of-focusinformation to the second information processing apparatus.

(22) The system of (21), wherein the second information processingapparatus is configured to: perform the predetermined processing; andtransmit a result of the predetermined processing to the firstinformation processing apparatus.

(23) An information processing system including: circuitry configured toidentify a first area of an image; acquire out-of-focus informationcorresponding to a second area of an image that is different from thefirst area of the image; and perform predetermined processing based onthe out-of-focus information.

(24) An information processing method including: performingpredetermined processing based on out-of-focus information correspondingto a first area of an image, the out-of-focus information correspondingto a difference in focus between the first area of the image and asecond area of the image that is different from the first area of theimage.

(25) A non-transitory computer-readable medium includingcomputer-program instructions, which when executed by a system, causethe system to: perform predetermined processing based on out-of-focusinformation corresponding to a first area of an image, the out-of-focusinformation corresponding to a difference in focus between the firstarea of the image and a second area of the image that is different fromthe first area of the image.

(26) An image processing device including:

an attention area determination unit configured to determine anattention area that is an area of an object of attention in an imageacquired by an imaging device; a blur information acquisition unitconfigured to acquire blur information of a non-attention area that isan area other than the attention area in the image; and a blur relatedprocessing unit configured to perform predetermined processing based onthe blur information.

(27) The image processing device according to (26), wherein the blurrelated processing unit enlarges an area including the attention areaand the non-attention area and displays the enlarged area in a displayunit in the image.

(28) The image processing device according to (26) or (27), wherein theblur related processing unit displays a blur adjustment user interfacefor adjusting the degree of blur in the image in the display unit.

(29) The image processing device according any of (26) to (28), whereinthe blur related processing unit applies blur processing to the image,and displays the present image and the image to which the blurprocessing has been applied in the display unit.

(30) The image processing device according to any of (26) to (29),wherein the blur related processing unit determines a composition of theimage, determines a recommended blur amount in the composition, andemphasizes the degree of blur in the non-attention area of an image tobe displayed in the display unit based on the recommended blur amount inthe composition of the image and the blur information of thenon-attention area of the image and displays the image.

(31) The image processing device according to any of (26) to (30),wherein the blur related processing unit determines a segmentation rangeof the image based on the blur information, and segments thesegmentation range from the image.

(32) The image processing device according to (31), wherein the blurrelated processing unit determines the segmentation range based on thedegree of separation of the blur information of the attention area andthe non-attention area in the segmentation range.

(33) The image processing device according to (31) or

(32), wherein, when there is a plurality of the segmentation ranges, theblur related processing unit segments an image with respect to each ofthe plurality of segmentation ranges, and generates a plurality ofsegmentation images.

(34) The image processing device according to any of (26) to (33),wherein the blur related processing unit performs image capturingoperation control that operates the imaging device in a bracket imagecapturing mode to acquire a plurality of images having differentdiaphragm aperture diameters by a single image capturing operation,based on the blur information.

(35) The image processing device according to (34), wherein, when theblur amount is a predetermined amount or less, the blur relatedprocessing unit operates the imaging device in the bracket imagecapturing mode in which the diaphragm aperture diameter is changed in adirection of becoming larger.

(36) The image processing device according to (34), wherein, when theblur amount is a predetermined amount or more, the blur relatedprocessing unit operates the imaging device in the bracket imagecapturing mode in which the diaphragm aperture diameter is changed in adirection of becoming smaller.

(37) The image processing device according to any of (26) to (36),wherein the blur related processing unit performs, according to the blurinformation of a captured image acquired by the imaging device, imageprocessing of emphasizing the degree of blur of the non-attention areaof the captured image.

(38) The image processing device according to any of (26) to (37),wherein the blur related processing unit displays a graph indicating theblur information in the display unit.

(39) The image processing device according to any of (26) to (38),wherein the blur related processing unit displays a histogram indicatingthe blur information in the display unit.

(40) The image processing device according to any of (26) to (39),wherein the blur information is an actual blur amount in thenon-attention area of the image.

(41) The image processing device according to any of (26) to (39),wherein the blur information is a blur prediction amount of thenon-attention area on an assumption that the imaging device has focusedthe attention area.

(42) A method of processing an image including: determining an attentionarea that is an area of an object of attention in an image acquired byan imaging device; acquiring blur information of a non-attention areathat is an area other than the attention area in the image; andperforming predetermined processing based on the blur information.

(43) An image processing program for causing a computer to execute amethod of processing an image, the method including: determining anattention area that is an area of an object of attention in an imageacquired by an imaging device; acquiring blur information of anon-attention area that is an area other than the attention area in theimage; and performing predetermined processing based on the blurinformation.

(44) An imaging device including: an image capturing unit configured toreceive light through an optical system, and to generate an image; anattention area determination unit configured to determine an attentionarea that is an area of an object of attention, in the image; a blurinformation acquisition unit configured to acquire blur information of anon-attention area that is an area other than the attention area in theimage; and a blur related processing unit configured to performpredetermined processing based on the blur information. It should beunderstood by those skilled in the art that various modifications,combinations, sub-combinations and alterations may occur depending ondesign requirements and other factors insofar as they are within thescope of the appended claims or the equivalents thereof. It should beunderstood by those skilled in the art that various modifications,combinations, sub-combinations and alterations may occur depending ondesign requirements and other factors insofar as they are within thescope of the appended claims or the equivalents thereof.

REFERENCE SIGNS LIST

-   -   1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 Image processing device    -   11 Attention area determination unit    -   12 Blur information acquisition unit    -   13, 23, 33, 43, 53, 63, 73, 83, 93, and 103 Blur related        processing unit    -   1007 Display unit    -   1003 Imaging element    -   1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, and    -   2000 Imaging device

The invention claimed is:
 1. An image capturing apparatus comprising: animaging element configured to generate an image; an image memory; andcircuitry configured to retrieve, in an absence of an image captureoperation, the image temporarily stored in the image memory, performpredetermined processing based on out-of-focus information correspondingto a first area of the image, the out-of-focus information correspondingto a difference in focus between the first area of the image and asecond area of the image that is different from the first area of theimage, perform the image capture operation after performing thepredetermined processing, and record a captured image based on the imagecapture operation that is performed after performing the predeterminedprocessing.
 2. The image capturing apparatus of claim 1, wherein thecircuitry is configured to: identify the first area of the image,determine first focus information associated with the first area,identify the second area of the image that is different from the firstarea of the image, determine second focus information associated withthe second area, and acquire the out-of-focus information correspondingto the first area of the image by determining the difference in focusbased on a difference between the first focus information and the secondfocus information.
 3. The image capturing apparatus of claim 2, whereinthe second area of the image is an in-focus area of the image.
 4. Theimage capturing apparatus of claim 3, wherein the predeterminedprocessing includes outputting a notification corresponding to theout-of-focus information.
 5. The image capturing apparatus of claim 2,wherein the predetermined processing includes controlling a display todisplay a notification indicating an out-of-focus amount based on theout-of-focus information.
 6. The image capturing apparatus of claim 5,wherein the circuitry is configured to control the display to displaythe image and the notification simultaneously.
 7. The image capturingapparatus of claim 5, wherein the circuitry is configured to control thedisplay to display the image with the notification superimposed on thedisplayed image.
 8. The image capturing apparatus of claim 1, whereinthe predetermined processing includes controlling a display to display abar graph indicating an out-of-focus amount based on the out-of-focusinformation.
 9. The image capturing apparatus of claim 8, wherein aheight of the bar graph corresponds to a range of the out-of-focusamount that may be realized based on at least one of the second area, afocal distance and a maximum aperture of a lens capturing the image. 10.The image capturing apparatus of claim 1, wherein the predeterminedprocessing includes controlling a display to display an enlarged versionof the image including the first area and the second area.
 11. The imagecapturing apparatus of claim 10, wherein an enlargement range of theenlarged version of the image is set so that a first display areacorresponding to the first area and a second display area correspondingto the second area are substantially similar.
 12. The image capturingapparatus of claim 10, wherein the circuitry is configured to controlthe display to display the image and the enlarged version of the imagesimultaneously.
 13. The image capturing apparatus of claim 10, whereinthe circuitry is configured to control the display to display theenlarged version of the image superimposed on the image.
 14. The imagecapturing apparatus of claim 1, wherein the predetermined processingincludes controlling a display to display the image while applying agraphical effect to the first area of the image.
 15. The image capturingapparatus of claim 1, wherein the predetermined processing includestrimming the image based on the out-of-focus information.
 16. The imagecapturing apparatus of claim 15, wherein the circuitry is configured tocontrol a display to display the trimmed image.
 17. The image capturingapparatus of claim 1, wherein the predetermined processing includesgenerating a first trimmed version of the image and a second trimmedversion of the image based on the out-of-focus information.
 18. Theimage capturing apparatus of claim 1, wherein the predeterminedprocessing includes generating a first trimmed version of the image anda second trimmed version of the image based on the out-of-focusinformation.
 19. The image capturing apparatus of claim 18, wherein thecircuitry is configured to control a display to display the firsttrimmed version of the image and the second trimmed version of theimage.
 20. An information processing method comprising: generating, withan imaging element, an image; retrieving, with circuitry and in anabsence of an image capture operation, the image that is temporarilystored in an image memory; identifying, with the circuitry, a first areaof the image; performing, with the circuitry, predetermined processingbased on out-of-focus information corresponding to the first area of theimage, the out-of-focus information corresponding to a difference infocus between the first area of the image and a second area of the imagethat is different from the first area of the image; performing, with thecircuitry, the image capture operation after performing thepredetermined processing; and recording, with the circuitry, a capturedimage based on the image capture operation that is performed afterperforming the predetermined processing.
 21. A non-transitorycomputer-readable medium including computer-program instructions that,when executed by an electronic processor, cause the electronic processorto perform a set of operations comprising: controlling an imagingelement to generate an image; retrieving, in an absence of an imagecapture operation, the image temporarily stored in an image memory;identifying a first area of the image; performing predeterminedprocessing based on out-of-focus information corresponding to the firstarea of the image, the out-of-focus information corresponding to adifference in focus between the first area of the image and a secondarea of the image that is different from the first area of the image;performing the image capture operation after performing thepredetermined processing; and recording a captured image based on theimage capture operation that is performed after performing thepredetermined processing.